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(54)Titte: UNSHARP MASKING USING CENTER WEIGHTED LOCAL VARLWCE FOR IMAGE SHARPENING 
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(57) Abstract 

A pseudo center weighted local variance in the neighbor- 
hood of an image pixel detennines the amplification factor 
multiplying the difference between the image pixel and its 
blurred counterpart before it is combined with the original im- 
age. The amplification factor is computed for each pixel in the 
image, and varies from a minimum value of about *1 when the 
center weighted variance is at a minimum value to some maxi- 
mum value when the center weighted variance reaches a maxi- 
mum value. The pseudo center weighted variance is computed 
by convolving the local neighborhood surrounding each image 
pixel with a Icemel of center weights having a maximum value 
at the center pixel. The pseudo center weighted variance is the 
difference between the.center weighted mean of the square of 
the local neighborhood of image pixels and the square of the 
center weighted mean. Both image sharpening and noise sup- 
pression is achieved in the same image by permitting the ampli- 
fication factor to vary between a fraction of unity and greater 
than unity as the center weighted variance varies from a low va- 
lue (uniform image) to a higher value (textured image). 
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UNSHARP MASKING USING 
CENTER WEIGHTED LOCAL VARIANCE 
FOR IMAGE SHARPENING AND NOISE SUPPRESSION 



05 



10 



BACKGROUND OF THE INVENTION 
Unsharp masking is a well-known technique 
for enhancing the sharpness of processed images. 
Specifically, an original digitized image is stored 
from which a blurred version may be created by 
convolving it with a low pass filter (such as an 
unsharp mask). The difference between the blurred 
image and its original is multiplied by a factor 
and the resulting product is added to the original 
15 image to generate a sharpened image. The problem is 
how to compute the factor k by which the difference 
image is multiplied. 

U.S. Patent No, 4,571,635 to Mahmoodi et 
al. teaches that the factor K is computed in 
20 accordance with a formula which depends upon the 
standard deviation of the image pixels in a local 
neighborhood. While this technique results in some 
improvement in the sharpness of the image, it would 
be desirable to obtain further improvement in image 
25 sharpness without proportionately amplifying the 
noise in uniform areas of the image. 

SUMMARY OF THE INVENTION 
A further significant increase in image 
sharpness is achieved without the expected 
30 proportionate increase in noise amplification in 
uniform areas of the image in the image process of 
the present invention, m the present invention, 
the difference image amplification factor K is not 
computed in accordance with the standard deviation 
35 as in Mahmoodi. instead it is computed in 
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accordance with a pseudo center weighted variance 
process. Alternatively, a sharpened center weighted 
variance process is used in the invention to compute 
the difference image multiplication factor K. while 
5 the idea of center weighting an ensemble of image 
pixels in a local neighborhood has been disclosed by 
Kato et al., U.S. Patent N*o. 4,315,318:, neither Kato 
et al. nor Mahmoodi et al. have considered such a 
computation in connection with the determination of 
10 the image difference multiplication factor K. 

PSEUDO CENTER WEIGHTED VARIANCE 
In the pseudo center weighted variance 
process of the invention, the difference image 
multiplication factor K is determined for each pixel 
xn the original image in accordance with a pseudo 
center weighted variance computed for all pixels in 
a predefined neighborhood surrounding the pixel of 
interest. Computation of the pseudo center weighted 
variance, in accordance with the invention, is as 
20 follows: The pixels in the neighborhood are 
individually squared and then convolved with a 
kernel having the same dimensionality as the 
neighborhood, the kernel comprising a mask of center 
weight coefficients whose maximum value lies i„ the 
25 center of the kernel. Further, each pixel in the 
neighborhood is convolved (without being squared) 
with the same kernel of center weighted 
coefficients, and the result of the latter 
convolution is then taken as a whole and squared, 
the results of the squaring then being subtracted 
from the previous convolution. The resulting 
difference is the pseudo center weighted variance. 
deLr' -Itiplication factor K is then 

determined m accordance with a function that 
increases as the pseudo center weighted variance 
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increases . 

Depending upon the steepness of the 
topology of the kernel of center weighted 
coefficients, the resulting sharpened image 
5 reproduces the edges in the original image with 

greater fidelity but without as much accentuation of 
noise in uniform image areas as might be expected. 
The image thus provides a significant improvement in 
the art. 

10 , DESCRIPTION OP THE DRAWINGS 

The invention may be understood by 
reference to the accompanying drawings of which: 

rig. 1 is a block diagram illustrating the 
system of the invention; 

^^9- 2a is a perspective view of a 
three-dimensional function defining the distribution 
Of center weighted weighting coefficients employed 
in the system of Fig. i; 

Pig. 2b is a diagram of a kernel of • 
20 weighting functions constructed from the 
three-dimensional function of Fig. 2a; 

Fig. 2c illustrates the use of the kernel 
of weighting functions of Fig. 2b in a large image 
array of pixels by the system of Pig. i; 
25 Figs. 3a and 3b illustrate alternative 

computations of the sharpening factor K as a 
function of the center weighted variance v in the 
system of Pig. i; 

Fig. 4 illustrates the distribution of 
variance values for uniform, textured and sharp 
image areas; and 

. Fig. 5 is a block diagram illustrating an 
alternative center weighted variance process which 
may be employed in the system of Pig. i. 
3.5 DETAILED DESCRIPTION 
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Ref erring to Fig. i, an original image 
array, comprising a plurality of digitized image 
pixels defining a two-dimensional image, is stored 
in a memory 10, each pixel Z being uniquely 
5 identified by a two-dimensional symbol z.. ... 
The unsharp masking process, in accordance with 
well-known techniques, employs a blurring processor 
50a which, in essence, averages each pixel received 
from the memory 10 with all of its neighbors in a 
10 small window of pre-defined size, typically li 

pixels by 11 pixels. The result is a blurred pixel 
which is then subtracted from the original pixel by 
a subtracter 20. The difference is then multiplied 
by a sharpening factor K at a multiplier 25, and the 
15 product is then added to the original image at an 
adder 30. The factor K is proportional to the 
center weighted variance of a small image 
neighborhood surrounding the pixel of interest. Ih 
local image neighborhoods having fairly large 
20 variances, K > o, and the result is a sharpened 
image pixel Z'..^^. which is stored in a memory 
35. After all of the image pixels stored in the 
memory lo have been thus processed, the array of 
image pixels in the memory 35 defines a much sharper 
25 image than the original array stored in the memory 
10. 

Reducing the foregoing to an algebraic 
expression, the system of Fig. 1. constructs the 
sharpened image Z' from the original and blurred 
images, z and. z respectively, as follows: 
(0) Z* - Z + K(Z - Z). 

In one aspect of the invention, it is not 
always desirable to sharpen the image, particularly 
32 m those image areas containing no sharp features, 
as this would simply boost the noise. The present 
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invention avoids such pitfalls. For example, in 
somewhat uniform image neighborhoods K may equal 0 
so that Z- » z in Equation (0), which avoids 
boosting noise because the output image Z' is simply 
5 the original image Z. in perfectly uniform image 

neighborhoods K may equal -1 so that Z* - Z in 
Equation (0), which achieves noise suppression 
because the output image Z' is simply the blurred 
10 image Z. The manner in which the invention provides 
both sharpening and noise suppression in different 
neighborhoods of the same image is described 
hereinbelow with respect to Figs. 3b and 3c. 

PSEWX? CRNTER WRTfiHTED VAPT^tar,; ; PRorKsp 

In the present invention, the sensitivity 
of the system of Fig. i to sharp features in the 
original image stored in the memory 10, and the 
fidelity with which such sharp features are 
reproduced by the system of Fig. i, are increased 
without (as would previously have been expected) 
unduly accentuating noise in uniform areas of the 
image (areas not containing sharp features). This 
is done by computing individual sharpening 
multiplier coefficients K.. ... for each image 
25 pixel Z^.^. processed in the system of Pig. i in 
accordance with the pseudo center weighted variance 
process. A pseudo center weighted variance 
processor 40 (hereinafter referred to as the pseudo 
processor 40) performs the computation. Such a 
30 processor may be a software- or firmware-programmed 
machine or may be a dedicated hard-wired processor. 

The pseudo processor 40 comprises two 
branches. In Fig. 1, the first branch comprises a 
square function 45 which computes the square of the 
35 value of each image pixel Z.,^^.. This branch 
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further comprises a blur convolver 50 and an 
associated center weight look-up table 55. The 
look-up table 55 stores a two-dimensional kernel of 
predetermined size (preferably 11 pixels by 11 
5 pixels) comprising weighting coefficients of which 
the centermost has the greatest amplitude while the 
outermost weighting coefficients have the smallest 
amplitudes, there being a continual variation 
therebetween. 

The topology defined by the weighting 
coefficients stored in the look-up table 55 is 
illustrated in Fig. 2a. The look-up table 55 stores 
an 11 X 11 kernel such as that illustrated in Fig. 
2b, comprising 11^ (or 121) weighting coefficients 
15 discretely defined at 121 points in an 11 x 11 

square array. The blur convolver 50 convolves an 11 
X- 11 neighborhood of image pixels taken from the 
original image array memory 10 surrounding the 
current pixel of interest, 2^.^^., with the 
20 corresponding weighting coefficients comprising the 
11 pixel by 11 pixel square window of weighting 
coefficients illustrated in Fig. 2b. The blur 
convolution performed by the blur convolver 50 
petforms the following process: 
25 i • +m, j • +m 

i-i'-m, j-j«-m =^'3 ^'3 " ' 

where N -.E. . and m - n=l, where n is the 
i»3 "2 

size of the kernel. (Note: n - li in the present 
30 example.) 

The foregoing process is repeated for each 
pixel in the original image array memory 10, so that- 
the kernel (or "window") of Fig. 2b is, in effect, 
moved across the two-dimensional image array so as 
35 to cover it entirely, one pixel at a time, in the 
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nianner illustrated in Fig. 2c. 

Each discrete summation computed by the 
blur convolver 50 is transmitted to the positive 
input of an adder 60. The negative input of the 
5 adder 60 receives a corresponding result computed in 
the other branch of the pseudo processor 40. 

The other branch of the pseudo processor 40 
includes a second blur convolver 50a and a center 
weight look-up table 55a, preferably identical to 

10 the first center weight look-up table 55. The blur 
convolver 50a operates in a manner identical to the 
blur convolver 50, except that it receives each 
image pixel Z^, .., from the original array memory 
10 without its being squared. Thus, the blur 

15 convolver 50a performs the same process except that 

it works on an array of unsguared image pixels 

Z^,.,. The blur convolver 50a thus performs the 

following operation: 

i'+m, j'+m 
20 2 w. .z. ./N 

2° i-i'-mTj-j'-m 

Each summation performed by the blur 
convolver 50a is then transmitted as a discrete 
value to a square function 45a, which functions in 
the manner identical to that of square function 45, 
in that it squares the value it receives. The 
output of the square function 45a is applied to the 
negative input of the adder 60 which then outputs, 
for each pixel Zj-^j. in the original image array, 
a pseudo center weighted variance V. . . . 

Prom the foregoing, it should be apparent 
that the output of the adder 60 is defined as 
follows: 

^ 'a - 

35 - 
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Each pseudo center weighted variance , . , is 
applied to a look-up table 70 which, in response, 
furnishes a corresponding sharpening multiplier 
coefficient K^.^j. to the input of the multiplier 
25. It should be noted that in the absence the 
centerweighting feature of the invention, all 
^i,^ ■ ^' snd the center weighted variance becomes 
a standard definition of variance. 

In accordance with, the invention, the 
look-up table 70 defines the multiplier coefficient 
K as a function of the pseudo . center weighted 
variance V, as illustrated in Pig. 3a. 
Alternatively, Fig. 3b illustrates a somewhat 
discontinuous way of defining K in terms of V, in 
that each knee in the curve is really a sharp 
comer. Other definitions of K as a function of V 
are perfectly acceptable within the scope of the 
invention, the central idea being that a greater 
variance justifies a greater sharpening of the image 
and therefore a larger value of the sharpening 
multiplier coefficient K. 

A fairly large kernel (11 x 11 pixels) is 
preferred to avoid blocking artifacts because the 
-effective- area in Fig. 2a is reduced due to 

the decrease in kernel amplitude at the outer pixels 
CALCULATION OF V^^ AND 

For a given input image or set of input 
images having the same noise characteristics, the 
variances and (defining the knees of the 
curves of Fig. 3) must first be calculated before 
the image is processed. To calculate V^^, a very 
uniform area in the image is chosen, and the 
variance value of each of the pixels in the uniform 
area is computed. is the average of all those 
variance values. 
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preferably lies between the typical 
variance value of teztured areas and the typical 
variance value of sharp edge areas in the same 
image. Fig. 4 is a graph showing variance values 
5 plotted against distribution (number of occurrences) 
for film grain noise (a "uniform" area) (Curve 14)^ 
texture (Curve 16), and sharp edge detail (Curve 18) 
for a typical digital image produced by scanning a 
photograph. The typical variance values of 

10 -texture" and "sharp edge" image areas correspond to 
the peaks of the curves 16 and 18 , respectively, in 
Fig. 4. is adjusted to find the most natural 

looking rendition of texture area. Typically, this 
adjustment is done by varying between the peak 

15 variance values of the curves 16 and 18 of Fig. 4 on 
a trial-by-error basis. Once this optimum value 

is found, it remains constant for the entire 
input image and subsequent input images of the same 
type. 

20 SHARPENING WITH NOISE SUPPRESSION 

Both image sharpening and noise suppression 
are achieved in different areas of the same image by 
employing the amplification factor function 
illustrated in Fig. 3b or 3c. Sharpening occurs for 
25 values of k > 0 while noise reduction occurs for 
values of -1 < k < 0. 

SHARP CENTER WEIGHTED VARIANCE PROCESS 
For even greater sensitivity to sharp 
features in the computation of the center weighted 
30 variant, a sharp center weighted variance processor 
40' (illustrated in Fig, 4) may be substituted for. 
the pseudo center weighted variance processor 40 in 
the system of Fig. 1. The sharp center weighted 
variance processor 40 • is virtually identical to the 
35 pseudo center weighted variance processor 40, except 
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that a square function 52 is interposed between the 
output of the center weight look-up table 55 and the 
blur convolver 50, so that each center weight 
coefficient ... is first squared, to become 
5 and the latter is then furnished to the 

blur convolver 50. (In a simpler embodiment, the 
square function 52 is implemented in the look-up 
table 55.) Thus, the sharp center weighted variance 
computed at the output of the adder 60 is as follows: 

Where « J w. 

The result is that the topalogy of the weighting 

15 coefficients employed in the blur convolver 50 of 
Fig. 4 is much sharper, corresponding to the dotted 
line topology suggested in Fig. 2a. 

In another variation on Fig. 4^ the square 
function 52 has the ability to raise each weighting 

20 coefficient W^^^, to any power between unity and 2 
(or any other power greater than unity). The power 
which the variable square function 52 employs is in 
turn determined by a computation of the pseudo 
variance by the pseudo variance processor 40^ each 

25 pseudo variance computed by the pseudo variance 

processor 40 being applied at a control input 52a of 
the variable square function 52. Preferably, for 
smaller values of the pseudo center weighted 
variance, the variable square function 52 raises 

30 to the minimum power (e.g., unity) while for 

larger values of the pseudo variance received at its 

control input 52a ^ the variable square function 52 

raises each center weighted coefficient W. • to 

i # J • 

the highest power. 
35 While the invention has been described in 
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detail with specific reference to the accompanying 
drawings/ variations and modifications may be made 
without departing from the spirit and scope of the 
invention. 
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CLAIMS: 

1. An image enhancement processing system " 
which receives an original image comprising an array 
of pixels, each pixel comprising an amplitude value, 
5 said system comprising: 

means for storing a mask of weighted 
coefficients characterized by a centermost 
coefficient of maximum amplitude and outermost 
coefficients of minimum amplitudes, said mask 
10 corresponding to a local neighborhood within said 
array of pixels; 

means for convolving said mask with local 
neighborhoods within said array of pixels so as to 
compute the center weighted variance of a local 
15 neighborhood surrounding each one of said pixels; 

means, responsive to said means for 
computing, for replacing the amplitude value of said 
one pixel with a new value, said new value 
corresponding to a new image sharpened with respect 
20 to said original image to a degree proportional to 
the amount by which said center weighted variance 
exceeds a first predetermined value; and 

means for storing said new value, whereby 
said means for storing stores a frame of new pixels 
25 representing an enhanced image after said means for 
computing and said means for replacing have 
processed all of the pixels in said array.. 

2. The system of claim 1 wherein said new 
value corresponds to a new image blurred with 
30 respect to said original image to a degree ^ 
proportional to the amount by which said center 
weighted variance is less than a second 5 
predetermined value less than said first 
predetermined value. 
35 3. The syistem of claim 1 wherein said 
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first predetermined value lies between a variance 
value typical of textured areas in said original 
image and a variance value typical of sharp edged 
areas in said original image. 
5 4. The system of claim 2 wherein said 

second predetermined value corresponds to the 
average variance of uniform areas in said original 
image. 

5. An image sharpening processor system, 
10 comprising: 

I. means for furnishing a plurality of 
digitized image pixels representing an image; 

II. image sharpening means, comprising: 

a) means for computing, for each pixel 
15 of said image, a blurred pixel comprising a low pass 

filtered value computed from a group of pixels in 
said image surrounding said one pixel, 

b) first differencing means for 
computing the difference between said one pixel and 

20 the corresponding blurred pixel, 

c) means for multiplying the 
difference computed by said first differencing means 
by a sharpening factor K to produce a product, 

d) adder means for adding said product 
25 to said corresponding blurred pixel to produce a 

sharpened pixel; 

III. means for computing a center weighted 
variance, comprising: 

a) means for computing a square of a 
30 center weighted mean of each pixel in said image, 
comprising: 

1) look-up table means storing an 
array of discrete weighting coefficients of which 
the centermost coefficient in said array has the 
35 greatest value while the outermost coefficients in 
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said array have the smallest values, 

2) convolver means operative for 
each one pixel in said images for selecting a 
neighborhood of pixels surrounding said one pixel in 

5 said image coincident with said array of weighting 
coefficients and for multiplying the corresponding 
pixels in said selected neighborhood by the 
corresponding weighting coefficients in said array* 
so as to produce a sum of products therefrom for all 
10 pixels in said selected neighborhood, 

3) first squaring means for 
computing the square of said first sum of products, 

b) means for computing . the center 
weighted mean of the square of each pixel, 
15 comprising: 

1) second squaring means for 
computing the square of each pixel in said image, 

2) convolver means for defining a 
neighborhood of squared pixels surrounding said one * 

20 squared pixel from said image array and for 

multiplying each pixel in said neighborhood by the 
corresponding weighting coefficient in said array of 
weighting coefficients so as to produce a second sum 
of products, 

25 c) second differencing means for 

subtracting said second sum of products from said 
first sum of products so as to produce a center 
weighted variance corresponding to said one pixel; 
IV. look-up table means defining said 

30 sharpening coefficient K as a function of said 
variance whereby said sharpening coefficient K 
increases as said variance increases over a finite 
range of said variance, including means for 
receiving the value of said variance corresponding 

35 to said one pixel and transmitting the corresponding 
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value of said sharpening coefficient K to said adder 
means . 

6, The system of claim 5 further 
comprising a third squaring function for computing 

5 the square of each of said weighting coefficients 
and then transmitting said square of said weighting 
coefficient to said second convolver means instead 
of said weighting coefficient. 

7. The system of claim 6 wherein said 
10 third squaring means comprises means for raising 

each of said weighting coefficients to a variable 
power, said power lying between unity and a selected 
number greater than unity in proportion to the 
magnitude received at a control input thereof, said 

15 system further comprising means for computing the 
variance of said one pixel and transmitting said 
variance to said control input whereby said variable 
power of said third squaring means increases as said 
variance increases. 

20 8. The system of claim 5 wherein said 

look-up table means defining said sharpening 
coefficient K as a function of said variance 
transmits a high value of K which results in maximum 
image sharpening in response to receiving a variance 

25 value at least nearly typical of sharp edge areas of 
said image and transmits a low value of K which 
results in maximum image blurring in response to 
receiving a variance value at least nearly typical 
of uniform areas of said image, whereby said system 

30 performs both image sharpening and noise suppression 
in the same image. 

9. The system of claim 8 wherein said high 
value of K is greater than unity while said low 
value of K is less than zero. 
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